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(54) User interface for Initiating a final scan using drag and drop 

(57) A user interface method for launching an opti- 
mized final scan of a selected region of interest selected 
from a preview scan of a document A user may drag 
the selected region of interest, presented in a preview 
scan of a document in a scanner window, and drop it on 
a software application, the desktop area (902), or a 
writeable folder, which launches an optimized final scan 
of the selected region of interest The image data result- 
ing from the optimized final scan automatically resides 
in the software application, the desktop area (902), or 
the writeable folder. In selecting a region of interest from 
the preview scan, scanner software (124) parameters 
are updated with information about the region of interest 
which optimize the final scan. The image data from the 
optimized final scan is then formatted in the format 
requested and delivered to the software application, the 
desktop area (902), or the writeable folder. 
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Description 
TECHNICAL FIELD 

[0001] This invention relates to document scanners 
and more particularly to the user interface for document 
scanners. Even more particularly, the invention relates 
to a user interlace that allows a user to automatically 
select a region of interest from a preview scan of a doc- 
ument and initiate an optimized final scan of the region 
of interest by moving it to an application using the drag 
and drop feature of the Windows operating system. 

BACKGROUND OF THE INVENTION 

[0002] Current scanner technology aDows a user to 
capture and utilize all or portions of various scanned 
documents, images, objects, eta for use within various 
computer applications, even when the documents are 
comprised of a variety of different components. A docu- 
ment containing text black and white and/or color pho- 
tographs, graphics, and color and black and white line 
art can be scanned in its entirety. Alternatively, the user 
may choose to select only certain portions of the origi- 
nal document for scanning by utilizing scanner software 
to select an area or particular image contained in a pre- 
view scan of the document. The selected area is then 
scanned to produce a final imaga 
[0003] The usability of existing scanning software has 
been hindered by the inability of users to easily move 
images from the scanner software to the user desktop 
or desired application, such as word processing or an 
Adobe PhotoShop m image editing type application. 
Typically, scanning software forces the user to either 
save the image as a ffle, copy the image to the clip- 
board, or return the data via a TWAIN or other industry 
standard inter-application communication protocol initi- 
ated scan. 

[0004] It is thus apparent that there is a need in the art 
for an improved method or apparatus which solves the 
objects of the invention. The present invention meets 
these and other needs in the art 

DISCLOSURE OF THE INVENTION 

[0005] rt is an aspect of the present invention to cfick 
on a selected region in a preview scan of a document 
and drag it to an open application or a desktop to launch 
an optimized final scan of the selected region. 
[0006] It is another aspect of the invention to have the 
image data resulting from the optimized final scan auto- 
matically reside in the open application or desktop. 
[0007] Yet another aspect of the invention is to update 
scanner software control parameters with information 
about the selected region to optimize the final scan of 
the selected region. 

[0008] Still another aspect of the invention is to offer 
different data formats for the optimized final scan based 



upon the selected region type. 
[0009] A further aspect of the invention is to allow the 
open application to query the scanner software to deter- 
mine if it can accept drop input of the selected region in 
5 an offered format before executing the optimized final 
scan. 

[0010] A stD I further aspect of the invention is to format 
the data from the optimized final scan into the format 
requested by the open application. 

w [0011] The above and other aspects of the invention 
are accomplished in a user interface for scanner soft- 
ware that allows a preview scan of a document to be 
presented to the user in a variable resolution preview 
window in a computer monitor. The user may then click 

is on a point within a region of interest in the preview scan 
data in the variable resolution preview window. Based 
on the characteristics of the data represented at the 
click point an area is determined that encompasses the 
dick point and the neighboring points that have similar 

20 characteristics to that of the click point This area is then 
classified by type based on the characteristics of the 
data represented within the area, such as text) gray 
scale image, color image, or black and white image. A 
selection marker is then automatically displayed around 

2s the area as a first estimate of what the user intended by 
making the mouse click at the click point 
[0012] The selection marker may be a bounding box 
rectangular in shape. For non-rectangular lasso" 
regions, image analysts software may be applied, auto- 

30 mat'cally tracing around the lasso region of interest. 
Portions of the scanned document that lie outside the 
selection marker are grayed out Based on the determi- 
nation of the type of area, adjustments are made auto- 
matically to data type, exposure, color, resolution, and 

35 sharpness settings that normally would have to be 
made manually by the user in prior art systems. 
[0013] If the selection marker automaticaDy generated 
does not represent the image of interest desired by the 
user, the user can adjust the size of the region of inter- 

40 est to include more area or include less area in several 
different ways. One such way is by dragging the selec- 
tion area bandies appropriately, expanding or contract- 
ing the selected area Or, the user may simultaneously 
click on a point and hold down a control key on a key- 

45 board, to either expand the region or image of interest to 
include additional area, or contract the region or image 
of interest to exclude area already selected. The user 
may also right click on the mouse to pop up a context 
menu, and select an expand selection option or select a 

so contract selection option from the menu. In addition, a 
different region of interest can be selected by the user 
by clicking in an unseiected area of the variable resolu- 
tion preview scan, and dragging the mouse to create a 
rectangular area bounded by a selection marker. 

55 [0014] Based on the type of image, windows having 
tools tor adjusting various aspects of the selected area 
are automatically enabled or cfisabled from user input 
Once the region of interest is property selected, and any 
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adjustments have been made, the user can use a 
mouse to drag and drop the selected area onto the 
desktop, a writeable folder, or onto an open application 
to launch an optimized final scan, or re-scan, of the 
selected area. After the optimized final scan, the result- 5 
ing image data resides in the desktop, folder, in a file, or 
within the open application. Alternatively, the user may 
utiGze pull down menus a buttons to launch the opti- 
mized final scan. However, if the pull down menu is 
used, save to fie, copy to clipboard, and print are the w 
only options available. Thus, the resulting image data 
wiD not automatically appear in an open application uti- 
lizing this option. 

DESCRIPTION OF THE DRAWINGS is 

[0015] The above and other aspects, features, and 
advantages of the invention will be better understood by 
reading the following more particular description of the 
invention, presented in conjunction with the following 20 
drawings, wherein: 

FIG. 1 shows a block diagram of a computer system 
incorporating the user interface for scanner soft- 
ware of the present invention; 25 
FIG. 2 shows a representation of the screen display 
of a computer monitor showing the user interface 
for scanner software of the present invention; 
FIG. 3A and Fig. 3B show a block diagram of the 
overall flow of the operation of the user interface for 30 
scanner software of the present invention; 
FIG. 4 shows a block diagram for determining the 
boundary of the region of interest and classifying 
the data type within the region of interest; 
FIG. 5 shows a block diagram for updating the 35 
screen tisplay of a computer monitor based on the 
selected area and its type; 
FIG. 6 shows a block diagram for performing an 
optimized final scan of the region of interest utilizing 
the drag and drop feature from the Windows oper- 40 
ating system; 

FIG. 7 shows a block diagram for re-sizing the 
boundary of a region of interest; 
FIG. 8A and FIG. 8B show a representation of the 
screen cfisplay of a computer monitor showing a 45 
screen capture of the user interface for scanner 
software of the present invention and an open appli- 
cation receiving a scanned image by way of drag 
and drop input; and 

FIG. 9A and FIG. 9B show a representation of the so 
screen cfisplay of a computer monitor showing a 
screen capture of the user interface for scanner 
software of the present invention and a desktop 
receiving a scanned image by way of drag and drop 
input 55 



BEST MODE FOR CARRYING OUT THE INVENTION 

[0016] The following description is of the best pres- 
ently contemplated mode of carrying out the present 
invention. This description is not to be taken in a limiting 
sense but is made merely for the purpose of descrfoing 
the general principles of the invention. The scope of the 
invention should be determined by referencing the 
appended claims. 

[0017] FIG. 1 shows a block diagram of a computer 
system incorporating the user interface for scanner soft- 
ware of the present invention. One skilled in the art will 
recognize that many other configurations are possible 
tor accessing a scanner with a computer system. Only 
one configuration is shown for simpfidty. Referring now 
to FIG 1, computer system 100 contains a processing 
element 102. Processing element 102 communicates 
with other elements of computer system 100 over a sys- 
tem bus 104. A keyboard 106 allows a user to input 
information into computer system 100 and a monitor 
110 allows computer system 100 to output information 
to the user. A graphical input device 108, commonly a 
mouse, is also used to input information. Scanner 
device 114 is also used to input information to computer 
system 100. 

[0018] Storage device 1 12 is used to store data and 
programs within computer system 100. Communica- 
tions interface 116, also connected to system bus 104, 
receives information from sources outside of computer 
system 100. A memory 118, also attached to system 
bus 104, contains an operating system 120, window 
manager 122, and scanner software 124 having the 
user interface for scanner software of the present inven- 
tion. In the preferred embodiment of the invention, oper- 
ating system 120 is the Microsoft Windows® operating 
system. Memory 1 18 also contains first software appli- 
cation 126 and second software application 128. One 
skilled in the art will recognize that many more software 
applications could reside in memory 118. Only two are 
shown for simplicity. 

[0019] FIG. 2 shows a representation of the screen 
display from monitor 110 (FIG. 1) showing the user 
interface for scanner software 124 (FIG. 1) of the 
present invention. Referring now to FIG. 2, scanner soft- 
ware window 200 shows a variable resolution preview 
window 202 containing variable resolution preview scan 
204 of a document that has been positioned on the flat- 
bed of scanner device 114 (FIG. 1). A user initiates a 
variable resolution preview scan 204 by pressing a pre- 
view scan mode button on scanner device 1 14 (FIG 1), 
or through a scan button or pull down menus presented 
to the user on monitor 1 10 through scanner software 
124, which sends a signal causing scanner device 114 
to use a preview scan mode to scan the document A 
preview scan mode is typically a low resolution scan, 
dependent on the resolution of the display and the size 
of the preview window. A user may also initiate a zoom 
scan of an area of the document positioned on the fiat- 
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bed, in which case the resolution of the data contained 
in variable resolution preview window 202 may be that 
of the scanner, which could be a considerably higher 
resolution than the preview scan mode. The data gener- 
ated from the variable resolution preview scan is cfis- 
played as variable resolution preview scan 204 in 
variable resolution preview window 202. Variable reso- 
lution preview scan 204 shews regions of various types, 
including text regions 206, black and white line art 
regions 208, color photo&aph regions 210, and (pay 
scale photograph region 212, which is partially superim- 
posed on one of the color photograph regions 21 0. 
[0020] A user has moved pointer 214 within gray scale 
photograph region 212 with graphical input device 108 
(FIG. 1) and has subsequently clicked on graphical 
input device 108. Scanner software 124 receives the 
click input, with pointer 214 in the shown location, and 
determines a boundary for the region surrounding the 
cfick point, and what data type of region is within the 
boundary. Alternatively, a user can accompBsh the 
above manually by moving pointer 214 to a corner of 
gray scale photograph region 212, and then cficking and 
dragging graphical input device 1 08 to encompass {pay 
scale photograph region 212 and then manually setting 
the data type via a pull down menu. In either case, vari- 
able resolution preview window 202 is updated with 
selection marker 216 drawn around the region identified 
as gray scale photograph region 212. In this example, 
selection marker 216 is a rectangular bounding box. 
The rest of the area of variable resolution preview scan 
204 outside of gray scale photograph region 212 is 
grayed out (not shown in FIG. 2). 
[0021] variable resolution preview window 202 also 
contains pull down menu bar 218 and tool bar 220, 
which provide the user with access to various functions 
of scanner software 124. ff no region has been selected, 
status bar 222 displays information regarding variable 
resolution preview scan 204 in Hs entirety, ff a region 
has been selected, as shown in FIG. 2, status bar 222 
displays current information regarding the region 
selected, which in this example is gray scale photo- 
graph region 212. Box 224 incficates that the region 
selected is a gray scale photographic image. Box 226 
incficates the image file size is 51 .7 KB. A scaling factor 
of 100% is shown in box 228. A different scaling factor 
may be displayed rf an inter-application communication 
link, such as TWAIN or OLE, has been established incfr- 
cating a preferred final size of the region of interest or if 
the user has specified one. Box 230 indicates that the 
selected image is measured in inches. The user may 
also make a "units" preference for Box 230 other than 
inches, such as centimeters, points, or pixels. Box 232 
incficates that the selected image is 1 .38 inches high, 
and box 234 indicates that the selected image is 1.74 
inches wide. Box 236 indicates a current resolution of 
150 dpt. 

[0022] Three floating windows are also shown in scan- 
ner software window 200 that present control tools to 



the user. The controls reflect adjustments made to vari- 
able resolution preview scan 204 based on the contents 
of the preview window. The user may use the tools pre- 
sented in the floating windows to further manipulate a 

5 selected region of interest 

[0023] Exposure adjustment window 238 offers con- 
trol tools that apply to all photographic output data types 
to preserve highlight and shadow detail Color adjust- 
ment window 240 applies only to color photographic 

10 output data types to adjust hue and saturation. Black 
and white threshold window 242 applies only to black 
and white binary output data types such as line art, clip 
art, halftones, and text 

[0024] FIG. 3A and FIG. 3B show a block diagram of 

is the overall flow of the operation of the user interface for 
scanner software of the present invention. The user 
interface is called from scanner software 124 (FIG. 1), 
when requested by the user of the scanner software 
124. Scanner software 124 performs other scanner 

20 functions which are not part of the invention. 

[0025] Referring now to FIG. 3A and FIG 3B, after 
entry, in step 300 a variable resolution preview scan is 
initiated for a document placed in the flatbed of scanner 
114 (FIG 1) that may contain text line art and/or color 

25 and black and white photographic images of interest. In 
step 302, variable resolution preview scan 204 (FIG 2) 
is displayed within variable resolution preview window 
202 (FIG. 2) in monitor 1 10 (FIG 1). 
[0026] Step 304 determines if click input from graphi- 

30 cal input device 108 (FIG. 1) on a region of interest 
within variable resolution preview scan 204 is received, 
or if a manual selection is made by clicking in an unse- 
lected area and dragging the mouse to create a rectan- 
gular selection area, or if an indication to quit the user 

35 interface is received, ff the latter is true, FIG. 3 returns 
to scanner software 124. ff a manual selection is made, 
then in step 305 the cfick and drag input establishing a 
boundary around a region of interest is received. Then 
in step 307 scanner software 124 performs dassrfica- 

40 tkxi analysis on the set of data elements contained 
within the boundary established by step 305, as more 
fully explained in FIG 4. 

[0027] ff click input is received in step 304, then in step 
306 click input on a region of interest is received by 
45 scanner software 1 24. Step 308 calls FIG. 4 which eval- 
uates the region surrounding the click point to deter- 
mine its boundary and its image data type. After 
returning from FIG 4, step 310 then calls FIG. 5 to 
update the screen display of monitor 110. After return- 
so ing from FIG. 5, step 312 determines if further input or 
an indication to quit the user interface, is received, ff the 
latter is true, FIG 3 returns to scanner software 124. ff 
the former is true, then step 314 determines what type 
of further input was received, ff the input received was a 
55 "mouse down" input signal selecting the region of inter- 
est fa a potential drag and drop from graphical input 
device 108, then step 316 calls FIG. 6 to potentially per- 
form an optimized final scan of the region of interest 
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After returning from FIG. 6, control returns to step 304 
where another region of interest may be selected, or 
input received to quit the user interface. 
[0028] if the input determined in step 314 was click 
input from graphical input device 108 on white space s 
within variable resolution preview scan 204, then in step 
318 scanner software 124 removes selection marker 
216 that was placed around the region of interest This 
action de-selects the current image selected. AD por- 
tions of variable resolution preview scan 204 that had 10 
been grayed out are restored and the display in variable 
resolution preview window 202 (FIG. 2) in monitor 110 is 
updated. Control then returns to step 304 where 
another region of interest may be selected, or input 
received to quit the user interface. is 
[0029] Finally, if the input determined in step 314 was 
click input on a region simultaneous with a control key 
held down, then step 320 caDs FIG. 7 to re-size the 
boundary of the region of interest. After returning from 
FIG. 7, control returns to step 31 0, which is a call to FIG. 20 
5 to update the screen display of monitor 110. 
[0030] FIG. 4 shows a block diagram for determining 
the boundary of the region of interest and classifying the 
data type within the region of interest Referring new to 
FIG. 4, after entry, scanner software 124 performs an 25 
analysis on the data elements from variable resolution 
preview scan 204 on the immediate region around the 
point clicked to determine the boundary of the region of 
interest and the data type within the region of interest In 
step 400, segmentation analysis is performed to deter- 30 
mine the boundary of the region of interest Various 
techniques are well known in the art for performing seg- 
mentation analysis, falling into three broad categories: 
top down strategy (model-driven), bottom up strategy 
(data-driven), and hybrid. For example, see Theo Pav- as 
lidis and Jiangying Zhou, Page Segmentation and 
Classification, published in Document Image Analysis, 
pp 226-238, (Lawrence CGorman and Rangachar Kas- 
turi, IEEE Press, 1995). Also, see Anil K. Jain and Bin 
Yu, Documentation Representation and Its Application 40 
to Page Decomposition, published in Pattern Analysis 
and Machine Intelligence, pp 294-308, (Vbtume 20, No. 
3, March 1998). In the preferred embodiment of the 
invention, a bottom up strategy is employed, utilizing the 
data element at the click point as the beginning of the 45 
segmentation extension analysis. However, segmenta- 
tion is not performed on the entire scanned document 
as is typically done, but is halted after the boundary for 
the region of interest surrounding the click point has 
been determined. The region of interest contains a set so 
of data elements from the variable resolution preview 
scan, and the boundary is determined by the set of con- 
nected outermost data elements. 
[0031 ] If the region of interest is rectangular in nature, 
extension is performed along a linear front resulting in ss 
a selection marker that is a rectangular bounding box. 
For non-rectangular lasso" regions of interest, exten- 
sion is performed along a non-linear front, resulting in a 



selection marker that is a "lasso* around the region of 
interest 

[0032] After performing segmentation analysis, in step 
402 scanner software 124 performs dassffication analy- 
sis on the set of data elements contained within the 
boundary established by step 400. Various techniques, 
also well known in the art for performing classification 
analysis, are also disclosed in the two articles cited 
abova In the preferred embodiment of the invention, the 
classification method used is that disclosed in Patent 
NO. 5,596,655 issued to Patricia D. Lopez on January 
21, 1997. Upon completion of classification analysis, 
FIG. 4 then returns to FIG. 3A. 
[0033] FIG. 5 shows a block diagram for updating the 
computer monitor display based on the selected area 
and its data type. Referring now to FIG. 5, in step 500 a 
selection marker is displayed in variable resolution pre- 
view scan 204 (FIG. 2) in monitor 110 (FIG. 1) around 
the region of interest determined from either FIG. 4 or 
from FIG. 7, which is discussed below. If a selection 
marker is already displayed when FIG. 5 is called, that 
selection marker is removed from display before dis- 
playing the next selection marker. In step 502 the 
remainder of variable resolution preview scan 204 lying 
outside of the region of interest bounded by the cur- 
rently displayed selection marker is grayed out 
[0034] Step 504 determines the data type of the 
region of interest If the data type in the region of inter- 
est is text or black and white line art then control 
passes to step 506 where scanner software 124 
updates the output in black and white threshold window 
242 (FIG. 2). In step 508 scanner software 124 updates 
the resolution, seating, output dimensions, and file size 
within status bar 222 (FIG. 2). In step 510 the controls in 
black and white threshold window 242 (FIG. 2) are ena- 
bled for user input. The user may make manual changes 
using the controls that have been enabled prior to initi- 
ating a re-scan of the selected region of interest Instep 
512 the controls for exposure adjustment window 238 
(FIG. 2) and color adjustment window 240 (FIG. 2) are 
disabled from user input and are grayed out. Thus, the 
steps in steps 506, 508, 510, and 512 automatically 
make many of the adjustments that the user would nor- 
mally have to make manually in other prior art scanning 
systems after selecting an image for scanning. The cur- 
rent invention reduces the complexity of the software 
and confusion of controls that the typical user may not 
understand. At this time, the user may also make man- 
ual changes using the controls that have been enabled 
prior to re-scarming the selected image. 
[0035] In step 504, if the data type in the region of 
interest is gray scale photographic, then control passes 
to step 514 where scanner software 124 changes the 
output in exposure adjustment window 238. In step 516 
scanner software 124 updates the resolution, scaling, 
output dimensions, and file size within status bar 222. In 
step 518 the controls in exposure adjustment window 
238 are enabled for user input The user may make 
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manual changes using the controls that have been ena- 
bled prior to initiating a re-scan of the selected region of 
interest In step 520 the controls in color adjustment 
window 240 and black and white threshold window 242 
are disabled from user input and are grayed out 
[0036] In step 504, if the data type in the region of 
interest is color photographic, then in step 522 scanner 
software 124 changes the output in exposure adjust- 
ment window 238 and in color adjustment window 240. 
In step 524 scanner software 124 updates the resolu- 
tion, scaling, output dimensions, and f Be size within sta- 
tus bar 222. In step 526 the controls in exposure 
adjustment window 238 and color adjustment window 
240 are enabled for user input. The user may make 
manual changes using the controls that have been ena- 
bled prior to initiating a re-scan of the selected region of 
interest In step 528 the controls in black and white 
threshold window 242 are disabled from user input and 
is grayed out After any of steps 512, 520, or 528, FIG. 
5 returns to FIG. 3. 

[0037] Thus, the steps in steps 506. 508, 510, and 
512; steps 514, 516, 518, and 520; and steps 522, 524, 
526, and 528, automatically make many of the adjust- 
ments that the user would normally have to make man- 
ually in other prior art scanning systems after selecting 
a region of interest for scanning. The current invention 
reduces the complexity of the software user interlace 
and confusion of controls that the typical user may not 
understand. 

[0038] FIG. 6 shows a block diagram for performing an 
optimized final scan of the selected region utilizing the 
drag and drop feature from the Windows operating sys- 
tem. Referring now to FIG. 6, in step 600, a data object 
is created and a list of one or more predetermined data 
formats provided in scanner software 124 is loaded into 
memory, based on the "mouse down" input signal 
selecting the region of interest displayed in monitor 110 
(FIG. 1) whose data type, or classification, was deter- 
mined in FIG. 4. In the preferred embodiment of the 
invention, for image types, such as true color, gray 
scale, palette, and black and white binary, a CF_DIB for- 
mat is provided, which is a Windows Device Independ- 
ent Bitmap, and a private drag/drop information format 
For metafile types, such as black and white scalable 
(vector), a CF_METAFILEPICT format is provided, 
which is a Windows Metafile Clipboard format and a 
private drag/drop information format For text, a 
CF_TEXT, which is ASCII Text, a CF_RTF which is Rich 
Text Format and a private drag/drop information format 
are provided. The private drag/drop information format 
is for internal use by scanner software 124 (FIG. 1), 
mainly as information for drawing feedback when the 
pointer 21 4 is moved over scanner software window 200 
(FIG. 2). 

[0039] In addition, two other data formats are provided 
for all data types when dragging and dropping to the 
desktop or to a folder. These two data formats. 
CFSTR_FILEDESCRIPTOR and 



CFSTR_FILECONTENTS are used by Explorer to cre- 
ate a He that win reside on the desktop or in a folder. 
The user must hold down a control key when the drag 
operation is initiated in order to add these data formats 
5 to the list of formats available to Explorer for a drop to 
the desktop or to a folder. 

[0040] Step 602 determines if a "mouse over input 
signal from graphical input device 108. which is handled 
by operating system 120 (FIG. 1) from a call made by 

io scanner software 124. is received identifying an open 
application in monitor 110. or identifying scanner soft- 
ware window 200 rtsetf ff the "mouse over" input signal 
is received identifying scanner software window 200. 
control passes to step 610. Step 610 then determines if 

15 a "mouse drop" input signal is received from graphical 
input device 108 selecting scanner software window 
200 displayed in monitor 110. This indicates that the 
user decided not to initiate an optimized final scan. tfthe 
answer is yes, then FIG. 6 returns to FIG. 3. ff the 

20 answer in step 610 is no, indicating no "mouse drop" 
signal was received, or that pointer 214 left scanner 
software window 200, then control returns to step 602. 
[0041] ff the "mouse over" input signal in step 602 is 
received identifying an open application, which was 

25 opened by a call to the software application, such as 
first software application 126 (FIG. 1), then operating 
system 120 notifies first software application 126 that 
pointer 214 has entered its window. Then in step 604 
first software application 1 26 queries the list of predeter- 

30 mined data formats associated with the data object cre- 
ated in step 600. ff one or more acceptable formats are 
found in the list generated in step 600, then first soft- 
ware application 126 lets operating system 120 know 
that it can receive the data and indicates a preferred for- 

35 mat from the one or more acceptable formats. Step 612 
then determines if a "mouse drop" input signal from 
graphical input device 108 is received, selecting first 
software application 126. ff the answer is no, indicating 
no "mouse drop" signal was received, or that pointer 

40 214 has left the first software application 126 window, 
then control returns to step 602. ff the answer in step 
612 is yes, indicating that a "mouse drop" input signal 
was received, then in step 614 first software application 
126 makes a call to operating system 120 and requests 

45 the image data in the format it prefers. Then, in step 
616, scanner software 124 sends a command to scan- 
ner 114 (FIG. 1) launching an optimized final scan of the 
document and the image data from the optimized final 
scan is put into the format requested. The image data 

so output from the optimized final scan for the region of 
interest portion of the document is sent to first software 
application 126, where the image data resides in the 
selected data format for further manipulation by the user 
within first software application 126. Control then 

55 returns to FIG. 3. 

[0042] ff in step 604 first software appfication 126 can- 
not find a format for the data that it can accept from the 
fist generated in step 600. then step 608 changes the 



6 



11 



EP0991 263A2 



12 



displayed pointer 21 4 to the universal "no" icon, a circle 
with a diagonal slash through it, indicating that a drop of 
the data object wifl not be allowed. Step 610 then deter- 
mines if a "mouse drop" input signal is received from 
graphical input device 108 on first software application 5 
126. If the answer is yes. then FIG. 6 returns to FIG. 3. 
If the answer in step 610 is no, then control returns to 
step 602. 

[0043] FIG. 7 shows a block diagram for re-sizing the 
boundary of a selected area. Referring now to FIG. 7, 10 
step 700 determines if the click input with graphical 
input device 108, simultaneous with a control key held 
down, was received within, or outside of, selection 
marker 216 (FIG. 2). If the dick input was within selec- 
tion marker 216, indicating that the user wants a smaller is 
area than what was automatically generated in FIG. 4, 
then step 704 finds a subset of the data elements within 
selection marker 216 immediately surrounding the click 
point, and establishes a new, smaller boundary around 
the subset of data elements. Then, in step 706, scanner 20 
software 124 performs classtfication analysts on the 
subset of data elements contained within the new 
boundary established by step 704. FIG. 7 then returns 
to FIG. 3. 

[0044] If in step 700 the click input with graphical input 2s 
device 108, simultaneous with a control key held down, 
was received outside of selection marker 216, indicating 
that the user wants a larger area than what was auto- 
matically generated in FIG. 4, control passes to step 
702. Step 702 determines if the click input with graphi- 30 
cal input device 108, simultaneous with a control key 
held down, was on white space or non-white space. If 
the click input was on white space, then step 712 
expands the boundary to encompass the entire 
scanned document FIG. 7 then returns to FIG. 3. 35 
[0045] If the click input in step 702 was on non-white 
space, then step 708 makes an adjustment to the 
parameters used in the segmentation analysis. Then 
step 710 calls FIG. 4 to evaluate the data elements sur- 
rounding the click point with the new parameters to 40 
determine a superset of data elements and a new 
boundary, and to determine the data type of the super- 
set of data elements within the new boundary. After 
returning from FIG. 4, FIG. 7 then returns to FIG. 3. 
[0046] FIG. 8A and FIG. 8B show a representation of 45 
the screen display of a computer monitor showing a 
screen capture of the user interface for scanner soft- 
ware of the present invention and an open application 
receiving a scanned image by way of drag and drop 
input. Referring now to FIGS. 8A and 8B, screen display so 
800 has scanner software window 200 (also shown in 
FIG. 2) and application software window 802 cBsplayed. 
Variable resolution preview scan 204 is displayed within 
variable resolution preview window 202 (also shown in 
FIG. 2). Black and white fine art region 208 (also shown ss 
in FIG. 2) has been selected by a user with graphical 
input device 108 (FIG. 1) by moving pointer 214 (FIG, 2) 
to a point within black and white line art region 208, and 



clicking graphical input device 108, causing selection 
marker 816 to be displayed around black and white Qne 
art region 208. Black and white line art region 208 con- 
tains black and white line art image 818. 
[Q047] Status bar 222 (also shown in FIG. 2) is 
updated to display current information regarding black 
and white fine art region 208. The three dialog boxes 
normally displayed in scanner software window 200 
adjacent to variable resolution preview window 202 
have been closed down by the user to create room in 
screen display 800 for application software window 802. 
[0048] First software application 1 26 (FIG. 1 ) is loaded 
into memory 118 (FIG. 1) and is displayed in application 
software window 802, which contains open work area 
820. First software application 126 may be a word 
processing application, a spread sheet app5catk>n, or a 
photo image editing type application that the user would 
like to bring a scanned version of black and white Gne 
art image 818 into. To accomplish this, the user moves 
pointer 214 (not shown in FIGS. 8A and 8B) with graph- 
ical input device 108 to a point within black and white 
line art region 208. The user then performs a drag and 
drop maneuver by holding down the graphical input 
device button, moving graphical input device 108 such 
that pointer 214 travels from the point within black and 
white line art region 208 displayed on monitor 110 that 
the "mouse down" input signal was sent, to any point 
within openwork area 820 within application software 
window 802, and then releasing the graphical input 
device button to send the mouse drop input signal 
selecting the open application. This drag and drop 
maneuver onto open work area 820 initiates within 
scanner software 124 (FIG. 1) an optimized final scan of 
the document which generates image data. When the 
optimized final scan is completed, the image data is 
sent from scanner 1 14 to firsts software application 126, 
where the image data appears as scanned image 822 in 
FIG. 8B. The image data fully resides within the applica- 
tion software. The user may then further manipulate 
scanned image 822 and/or save the image data to a file. 
[0049] If the user selects a text region in variable res- 
olution preview scan 204 to drag and drop to application 
software window 802, the text region is scanned utilizing 
the automatic adjustments and updates made by scan- 
ner software 124 after the text region was selected. 
OCR, Optical Character Recognition, is then performed. 
The resulting data is put into ASCII Text format or Rich 
Text Format for delivery to the open appfication. 
[00501 FK3L 9A and FIG. 9B show a representation of 
the screen display of a computer monitor showing a 
screen capture of the user interface for scanner soft- 
ware of the present invention and a desktop receiving a 
scanned image using the drag and drop feature of the 
Windows operating system. Referring now to FIGS. 9A 
and 9B, screen display 900 has scanner software win- 
dow 200 (also shown in FIG. 2) and desktop area 902 
displayed. Variable resolution preview scan 204 is dis- 
played within variable resolution preview window 202 
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(also shown in FIG. 2). Black and white line art region 
208 (also shown in FIG. 2) has been selected by a user 
with cpaphical input device 108 (FIG. 1) by moving 
pointer 21 4 (FIG. 2) to a point within black and white line 
art region 208, and clicking graphical input device 108, 
causing selection marker 816 to be displayed around 
black and white line art region 208. Black and white line 
art region 208 contains black and white Brie art image 
818. 

[0051] Status bar 222 (also shown in FIGl 2) is 
updated to display current information regarding black 
and white line art region 208. The three floating win- 
dows normally displayed in scanner software window 
200 adjacent to variable resolution preview window 202 
have been closed down by the user to create room in 
screen display 900 for desktop area 902. 
[0052] The user may bring a scanned version of black 
and white line art image 818 onto desktop area 902. To 
accomplish this, the user moves pointer 214 with graph- 
ical input device 108 to a point within black and white 
line art region 208. The user then performs a drag and 
drop maneuver. A control key must be held down when 
the drag operation is initiated to add the data formats to 
the list of formats available to Explorer for a drop to the 
desktop or to a folder. The graphical input device button 
is held down, then graphical input device 108 is moved 
such that pointer 214 (not shown in FIGS 9A or 9B) trav- 
els from the "mouse down" point within black and white 
line art region 208 to any point within desktop area 902, 
and then the graphical input device button is released, 
completing the cfrag and drop maneuver. This same 
procedure may be applied to bring a scanned version of 
black and white line art image 818 to a writeable folder 
within a directory window. This drag and drop maneuver 
onto desktop area 902, or onto a writeable folder within 
a directory window (not shown in FIGS. 9A and 9B) ini- 
tiates within scanner software 124 (FIG. 1) an optimized 
final scan of the document which generates image data 
When the optimized final scan is completed, the image 
data is sent form scanner 114 to the desktop, which is 
an open application, where the image data appears in 
FIG. 8B as scan.bmp icon 904, representing a ffle cre- 
ated by Explorer, the file manager software in Windows, 
upon receiving the image data from scanner 114. The 
same icon would appear for a drop to a writeable folder 
within a directory window. The user may then further 
manipulate the image with a software application that 
can accept ixnp file formats. 
[0053] If the user selects a text region in variable res- 
olution preview scan 204 to drag and drop to desktop 
area 902, the text region is scanned utilizing the auto- 
matic adjustments and updates made by scanner soft- 
ware 124 after the text region was selected. OCR. 
Optical Character Recognition, is then performed. The 
resulting data is put into ASCII Text format for defivery to 
desktop area 902. A text file is created by Explorer, the 
file manager software in Microsoft Windows ®, upon 
receiving the image data from scanner software 124, 



and the file appears as an icon in open work area 820, 
which may be opened by Notepad or whatever other 
comparable software has been set up by the user. 
[0054] Having described a presently preferred embod- 

5 iment of the present invention, it will be understood by 
those skilled in the art that many changes in construc- 
tion and circuitry and widely differing embodiments and 
applications of the invention wiD suggest themselves 
without departing from the scope of the present inven- 

io tion, as defined in the claims. The cfisdosures and the 
description herein are intended to be illustrative and are 
not in any sense limiting of the invention, defined in 
scope by the following claims. 

is Claims 

1. A user interface method, within a computer system 
(1 00), for transferring image data from scanner soft- 
ware (124) to a software application, said method 
20 comprising: 

(a) launching (300) a preview scan of a docu- 
ment, with said scanner software (124) loaded 
into a memory (118) of a computer system 

25 (100), creating preview scan data of said docu- 

ment; 

(b) displaying (302) said preview scan data on 
a monitor (110) connected to said computer 
system (100); 

30 (c) selecting a region of interest from said pre- 

view scan data displayed on said monitor 
(110); 

(d) receiving (314), into said memory (118) in 
said computer system (100), a mouse down 

35 input signal from a graphical input device (108) 

connected to said computer system (100), 
wherein said mouse down input signal selects 
(600) said selected region of interest within 
said preview scan data; 

40 (e) receiving (612), into said memory (118) in 

said computer system (100), a mouse drop 
input signal from said graphical input device 
(108), wherein said mouse drop input signal 
selects said software application, wherein said 

45 software application was previously loaded into 

said memory (118) in said computer system 
(100) and displayed in an open application win- 
dow (802) on said monitor (110); 

(f) launching (616), with said scanner software 
so (124), a final scan of said document creating 

said image data, wherein said image data of 
said document corresponds to said selected 
region of interest; 

(g) receiving (616) said image data in said 
55 memory(118) in said computer system (100); 

and 

(h) sending (616) said image data to said 
selected software appfication. 
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2. A user interface method, within a computer system 
(100), for transferring data from scanner software 
(124) to a software application according to claim 1 
wherein step (a) further comprises step (aO) per- 
formed before step (a), and step (f) further com- 5 
prises step (fl): 



3. A user interface method, within a computer system 
(100), for transferring data from scanner software 
(124) to a software application according to claim 1 
wherein step (d) further comprises steps (d1) and 20 
(d2): 



4. A user interface method, within a computer system 30 
(100), for transferring data from scanner software 
(124) to a software application according to claim 3 
wherein when said data type for said selected 
region of interest is true color, gray scale, palette, or 
black and white binary, said data format is a Win- 35 
dows Device Independent Bitmap format. 

5. A user interface method, within a computer system 
(100), for transferring data from scanner software 
(124) to a software application according to claim 3 40 
wherein when said data type for said selected 
region of interest is black and white scalable vector, 
said data format is a Windows Metafile Clipboard 
format 

45 

6. A user interface method, within a computer system 
(100), for transferring data from scanner software 
(1 24) to a software application according to claim 3 
wherein when said data type for said selected 
region of interest is text, said data format is an so 
ASCII Text format or a Rich Text format 

7. A user interface method, within a computer system 
(100), for transferring data from scanner software 

(1 24) to a software application according to claim 3 55 
wherein step (e) further comprises the following 
steps (eOa) and (eOb) performed before step (e) 
and step (e1) performed after step (e), and step (g) 



further comprises step (gO) performed before step 
(9): 

(eOa) receiving (602) a mouse over input signal 
from said graphical input device (108) identify- 
ing said software application displayed on said 
monitor (110); 

(eOb) querying (604) by said software applica- 
tion if said scanner software (124) offers one of 
said at least one data format associated with 
said data type of said selected region of inter- 
est, wherein said software application can 
accept drop input of said data type formatted 
into said one of said at least one data format 
associated with said data type; 
(e1) requesting (614), by said software applica- 
tion, that said image data be formatted into said 
one of said at least one data format associated 
with said data type that said software applica- 
tion can accept; and 

(gO) formatting (616) said image data into said 
one of said at least one data format associated 
with said data type that said software applica- 
tion can accept. 

8. A user interface method, within a computer system 
(100), for transferring data from scanner software 
(124) to a software application according to claim 1 
further comprising the steps of: 

(i) receiving (616) said image data in said 
selected software application in said memory 
(118); and 

(j) cfisplaying (616) said image data in said 
open application window (802) on said monitor 
(110). 

9. A user interface method, within a computer system 
(100), for transferring data from scanner software 
(124) to a software application according to claim 8 
wherein said software application is a desktop area 
(902) and wherein step (i) further comprises steps 
(i1) through (i3): 

(11) creating a file, by file manager software 
1 within an operating system loaded into said 

memory (1 18), upon receiving said image data 
into said memory (118), wherein a file type of 
said ffleis determined based on said data type; 

(12) saving said file as said file type in said com- 
puter system (100); and 

(13) displaying a representation of said He of 
said file type as an icon on said desktop area 
(902). 

10. A user interface method, within a computer system 
(100), for transferring data from scanner software 
(124) to a software application according to claim 8 



(aO) placing said document into a position to be 
scanned by a scanner device (1 14) connected 
to said computer system (100); and 10 
(f1) receiving in said scanner device (114), 
from said scanner software (124), a command 
to launch said final scan of said document 
wherein said final scan creates said image 
data. is 



(d1) receiving (600) a data type for said 
selected region of interest in said memory 
(1 18) in said computer system (100); and 25 
(d2) retrieving (616) at least one data format 
associated with said data type for said selected 
region of interest 
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wherein said software application is a writeaUe 
folder and wherein step (i) further comprises steps 
(i1) through (©): 

(11) creating a file, by file manager software s 
within an operating system loaded into said 
memory (118). upon receiving said image data 
into said memory (1 18), wherein a type of said 
fOe is determined based on said data type; 

(12) saving said f De as said file type in said com- w 
puter system (100); and 

(13) displaying a representation of said file as 
an icon on said writeable folder. 
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